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ABSTRACT 

An  efficient  implementation  of  Ada  task  termination  for  shared  memory  MTMD 
architectures  is  presented.  The  solution  is  highly  parallel  and  distributed;  termina- 
tion is  not  synchronized  by  a  centralized  supervisor,  and  there  are  no  critical  sec- 
tions. The  handling  of  block  and  subprogram  termination  that  depend  on  nested 
tasks  is  also  included.  This  implementation  can  be  easily  adapted  to  message  based 
systems. 

1.  Introduction 

An  efficient  implementation  of  Ada  tasking  in  a  multiprocessor  environment  must 
contend  with  several  potentially  expensive  serialization  points,  that  is  to  say,  constructs 
whose  execution  may  take  a  time  proportional  to  the  number  of  tasks  present  Such  seri- 
alization pHDints  negate  the  efficiency  that  can  otherwise  be  obtained  by  multitasking.  In 
proposed  parallel  architectures  with  large  numbers  of  processors  ([GLR83],  [Pfi85]), 
where  gains  from  paralleUzation  will  be  substantial,  it  is  particularly  important  to  minim- 
ize serial  bottlenecks.  In  this  paper  we  examine  one  such  synchronization  problem,  the 
efficient  implementation  of  the  termination  rules  for  Ada  tasks. 

Task  termination  impxjses  a  synchronization  point  on  several  language  constructs. 
In  accord  with  the  block  structure  discipline  of  Ada,  the  end  of  a  block,  subprogram,  or 
task  in  which  tasks  have  been  declared  is  a  synchronization  point:  execution  of  the 
construct's  outer  context  may  not  proceed  until  all  such  tasks  have  terminated.  Termina- 
tion is  complicated  by  the  terminate  alternative  feature  of  select  statements.  Because  of 
this  feature,  sets  of  tasks  must  be  terminated  together  based  on  the  state  of  all  of  the  tasks 
in  the  set.  Therefore,  many  implementations  in  a  single  processor  environment  rely  on  a 
centralized  supervisor  ([Ros83],  [RB84])  to  detect  terminatability  and  to  lock  out  other 
tasks  during  task  termination  synchronization.  More  complex  solutions  to  task  termina- 
tion have  been  presented  for  distributed,  nonshared  memor>'  machines  ([Kaf85], 
[FW86]).  The  solutions  in  ([Kaf85],  [FW86])  involve  message  passing  among  sets  of 
tasks  to  achieve  the  effects  of  a  centralized  supervisor;  for  example,  a  task  must  some- 
times exchange  messages  with  its  parent  to  determine  whether  to  engage  in  a  rendezvous. 
Related  work,  not  directly  involving  Ada,  is  presented  in  [CL82].  An  algorithm  is  given 
for  distributed  termination  in  systems  with  dynamic  creation  of  tasks,  v^ith  the  restriction 
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that  tasks  must  know  with  whom  they  can  communicate  when  they  are  created. 

Little  has  been  pubhshed  describing  task  termination  on  MIMD,  shared  memory 
machines,  an  architecture  for  which  Ada  tasking  is  well  suited  ([SS85]).  The  solution 
presented  here  is  for  a  shared  memory  architecture  (in  particular,  the  Ultracomputer 
[GLR83]),  but  can  be  readily  adapted  to  simplify  the  message  based  approach  as  well.  It 
generalizes  the  single  processor  approach  described  in  [RB84]  to  a  multiprocessor 
environment  by  eliminating  the  centralized  supervisor  and  distributing  its  work  among 
the  synchronizing  tasks.  We  prove  the  correctness  of  our  implementation  by  showing 
that  the  critical  sections  provided  by  a  centralized  supervisor  are  not  required.  We  nei- 
ther employ  a  complex  locking  mechanism  nor  rely  on  extensive  communication  among 
tasks.  Task  synchronization  is  for  the  most  part  local,  i.e.  between  task  and  subtask. 
Finally,  the  implementation  presented  is  complete  in  that  it  includes  the  handling  of 
block  and  subprogram  termination  which  depend  on  nested  task  termination. 

The  simplicity  of  the  distributed  solution  is  surprising  in  view  of  the  richness  of  Ada 
tasking  features.  The  inclusion  of  task  types  and  access  to  task  types  in  ANSI/MIL- 
STD-1815  Ada,  while  providing  an  orthogonal  and  powerful  language  construct,  seem- 
ingly complicates  questions  of  distributed  termination.  For  example,  the  claim  in 
[Cle82],  that  only  the  parent  and  siblings  of  a  task  waiting  at  a  terminate  alternative  are 
able  to  call  one  of  its  entries,  does  not  hold  in  ANSI/Ada.  Because  of  the  existence  of 
access  task  types,  tasks  can  be  dynamically  created  using  an  allocator,  and  p)ointers  to 
tasks  can  be  freely  assigned  to  variables  and  transmitted  as  actual  parameters  in  calls. 

Fortunately,  Ada  termination  rules,  described  in  the  next  section,  allow  a  straightforward 
distributed  implementation  in  spite  of  these  complications. 

2.  Ada  Task  Termination  Rules 

The  full  description  of  Ada  task  termination  rules  is  given  in  [LRM83]  9.4.  We 
give  a  brief  summary  of  these  rules  here,  and  illustrate  them  with  a  few  examples. 

Tasks  are  directly  dependent  on  master  constructs,  where  a  master  construct  m  is 
either  another  task,  a  block,  a  subprogram,  or  a  library  package.  To  precisely  define 
direct  dependency,  we  must  consider  two  cases: 

(1)  a  task  created  by  the  evaluation  of  an  allocator  depends  directly  on  the  master 
that  elaborates  the  corresponding  access  type  definition; 

(2)  otherwise,  a  task  depends  directly  on  the  master  whose  execution  creates  the  task 
object. 

If  the  direct  master  of  a  task  ;  is  a  block,  then  t  is  said  to  be  an  inner  dependent  of 
the  task  that  is  executing  the  block.  Such  a  task,  in  effect,  has  two  masters:  its  direct 
master,  which  is  a  block,  will  be  referred  to  as  its  master  block;  an  indirect  master,  which 
is  the  task  executing  the  block,  will  be  referred  to  as  its  master  task.  This  distinction  is 
essential  to  the  algorithms  presented  below.  If  the  direct  master  of  a  task  r  is  another 
task,  then  t  has  a  master  task,  but  no  master  block.  If  the  direct  master  of  r  is  a  subpro- 
gram, then  the  subprogram  will  also  be  referred  to  as  the  master  block  of  r.  In  this  case,  t 
has  a  master  block,  but  no  master  task. 

The  notion  of  dependence  is  generalized  as  follows:  a  task  t  depends  on  a  master 
construct  m  if  it  directly  depends  on  m,  is  an  inner  dependent  of  m,  or  depends  on  another 


master  m'  that  depends  on  m. 

Task  termination  then  takes  place  under  the  following  conditions.  If  a  task  has  no 
dependent  tasks,  it  terminates  when  it  has  completed,  i.e.  when  it  has  finished  executing 
its  statements.  If  a  task  has  dependent  tasks,  it  terminates  when  it  has  completed,  and  all 
of  its  dependents  have  terminated. 

A  block  or  subprogram  that  has  dependent  tasks  can  only  be  left  when  all  of  its 
dependents  have  terminated.  For  consistency  we  will  say  that  a  block  or  subprogram  ter- 
minates when  it  is  left. 

This  straightforward  rule  is  complicated  by  the  Ada  terminate  alternative,  which 
provides  another  way  in  which  a  task  may  terminate.  A  task  terminates  when  its  execu- 
tion has  reached  an  open  terminate  alternative  in  a  select  statement,  and  the  following 
conditions  are  satisfied: 

(1)  The  task  depends  on  some  master  whose  execution  is  completed  (hence  not  a 
library  package). 

(2)  Each  task  that  depends  on  the  master  considered  is  either  already  terminated  or 
similarly  waiting  on  an  open  terminate  alternative  of  a  select  statement. 

When  both  conditions  are  satisfied,  the  task  considered  becomes  terminated, 
together  with  all  tasks  that  depend  on  its  master.  We  call  the  termination  of  a  task  and 
all  of  its  dependents  the  termination  wave  of  the  task.  Tasks  that  depend  on  library  pack- 
ages are  a  special  case,  and  are  discussed  in  the  Appendix. 

Some  of  the  implications  of  these  rules  are  demonstrated  in  the  following  examples. 
It  is  useful  to  represent  the  dependency  relation  among  master  tasks  and  dependent  tasks 
by  trees.  The  root  node  of  a  dependency  tree  is  a  master  task  or  a  master  block;  the  chil- 
dren of  a  tree  node  are  all  the  tasks  that  depend  on  the  node.  (A  master  block  cannot  be 
an  interior  node  in  the  dependency  trees,  since  a  master  block  is  not  a  dependent  of  any 
other  construct.  A  task  t  that  has  both  a  master  task  and  a  master  block  is  an  interior  node 
in  two  trees.)  After  each  example,  we  give  its  dependency  tree(s). 

Example  1  -  Tasks  With  Terminate  Alternatives 

This  example  illustrates  nested  tasks  with  terminate  alternatives  in  select  state- 
ments. A  portion  of  the  dependency  tree  from  Example  1  is  shown  in  Figure  1.  If  all  of 
the  tasks  in  the  tree  are  executing  their  select  loops,  termination  can  take  place  only  if  all 
of  the  child  and  grandchild  tasks  are  waiting  at  terminate  alternatives.  All  of  the  tasks  in 
the  tree  will  terminate  together  in  a  single  termination  wave.  Note  that  a  given  child  task 
may  be  called  by  any  of  the  15  grandchild  tasks,  and  that  even  if  all  of  the  child  tasks  are 
waiting  at  terminate  alternatives,  the  state  of  a  child  may  change  because  another  task 
(e.g.  a  grandchild)  may  invoke  it,  and  so  termination  may  not  be  possible. 


master 


dependence: 
entry  call: 


Figure  1 


grandchild 


task  master; 

task  body  master  is 

task  type  children  is 

entry  el(i:  integer); 
end  children; 

child:  array  (1..5)  of  children; 
—  Other  declarations. 

task  body  children  is 

task  type  grandchildren  is 

entry  e2(j:  integer); 
end  grandchildren; 


grandchild:  array  (1.. 3)  of  grandchildren; 
-  Other  declarations. 

task  body  grandchildren  is 
begin  -  Code  executed  by  grandchild. 
loop 
select 

accept  e2(j:  integer)  do 

chiida).ei(j); 
end  e2; 

or 

terminate; 
end  select; 
end  loop; 
end  grandchildren; 

begin  —  Code  executed  by  child. 
loop 
select 

accept  el(i:  integer)  do 

end  el; 
or 

terminate; 
end  select; 
end  loop; 
end  children; 

begin 

~  Code  body  of  master. 

end  master; 

Example  2  -  Tasks  That  Exchange  Tasks  As  Entry  Parameters 

A  task  can  also  be  called  by  a  task  that  is  not  a  dependent  of  its  master,  as  illustrated 
below.  The  dependency  tree  for  the  program  is  shown  in  Figure  2. 
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middleman 


called 


Figure  2 

task  master; 

task  body  master  is 

task  middleman  is 

entry  el (i:  integer); 
end  middleman; 

task  type  called_type  is 
entry  e2(j:  integer); 
end  called_type; 

task  caller  is 

entry  e(tasktocall:  in  called_type); 
end  caller; 

task  body  called_type  is 
begin 
loop 
select 

accept  e2(j:  integer)  do 

end  e2; 


or 

terminate; 
end  select; 
end  loop; 
end  called_type; 


task  body  middleman  is 

called  :  called_type;  -  Create  dependent  task. 
begin 

caller.e(called);  --  Pass  dependent  task  to  caller. 
loop 
select 

accept  el(i:  integer)  do 

end  el; 

or 

terminate; 
end  select; 
end  loop; 

end  middleman; 

task  body  caller  is 
begin 

accept  e(tasktocall:  in  called_type)  do 
tasktocall.e2(17); 

end  e; 

end  caller; 

—  Code  body  of  master. 

end  master; 


m    master 


Example  3  -  Task  With  Allocators 

This  example  illustrates  dynamic  task  allocation.  The  dependency  tree  is  shown  in 
Figure  3.  After  the  task  dependent  is  activated,  it  dynamically  creates  tasks  tJ  and  t2. 
These  dynamically  created  tasks  are  dependents  of  dependent  because  their  access  type  is 


defined  in  dependent.  Additionally,  every  time  the  entry  e  is  called  and  the  rendezvous  is 
complete  (e.g.  see  the  body  of  master),  dependent  creates  a  new  task  t3,  also  a  direct 
dependent  of  dependent.  Termination  can  take  place  when  dependent  is  waiting  at  a  ter- 
minate alternative,  all  tasks  created  by  dynamic  allocators  have  terminated,  and  master  is 
complete.  Note  that,  depending  on  how  many  times  the  entry  e  is  called,  an  arbitrary 
number  of  subtasks  t3  may  have  been  generated. 

task  master; 

task  body  master  is 

task  dependent  is 
entry  e(i:  integer); 

end  dependent; 

task  type  dynamic; 
task  body  dynamic  is 
begin 

end  dynamic; 

task  body  dependent  is 

type  dynamic_ptr  is  access  dynamic; 
tl,  t2,  t3:  dynamic_ptr, 
l)egin 

tl  :=  new  dynamic; 
t2  :=  new  dynamic; 
loop 
select 

accept  e(i:  integer)  do 
t3  :=  new  dynamic; 

end  e; 
or 

terminate; 
end  select; 
end  loop; 
end  dependent; 

begin  --  Body  of  master. 

dependent. e(7); 
end  master; 

Example  4  -  Master  Blocks 

This  example  illustrates  tasks  dependent  on  master  blocks,  and  some  of  the  compli- 
cations that  derive  from  dependency  rules.  Figure  4  shows  the  dependency  trees  of  the 
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inner 


• 
t2 


inner  dependence: 


Figure  4 


master  task  and  the  master  block.  The  declaration  of  task  tl  is  elaborated  by  task  master, 
so  tl  is  a  direct  dependent  of  master.  Master  contains  an  inner  block:  since  the  declara- 
tion of  task  t2  is  elaborated  in  the  block,  the  block  is  the  direct  master  of  t2.  Task  t2  is  an 
inner  dependent  of  master.  Task  t3  is  also  created  via  an  allocator  within  the  inner  block. 
However,  since  the  access  type  of  t3  is  declared  in  master,  (see  rule  (1),  Section  2),  t3  is 
the  direct  dependent  of  master. 

The  inner  block  can  exit  when  its  dependent  i2  terminates  or  waits  on  a  terminate 
alternative  and  the  block  completes.  Note,  however,  that  the  block  itself  contains  a  select 
loop  with  a  terminate  alternative.  If  the  select  loop  is  executed,  it  is  possible  that  the 
inner  block  may  never  complete.  In  this  case  the  task  master  (not  the  block)  is  waiting  to 
terminate:  master  may  then  terminate  according  to  the  rules  defined  above,  and  when  the 
direct  and  inner  dependents  of  master,  namely  tl,  t2,  and  t3,  have  terminated  or  are  wait- 
ing to  terminate.  For  this  reason,  task  t2  is  in  the  dependency  trees  of  both  master  and 
the  inner  block. 

task  master  is 
entry  em; 

end  master; 

task  body  master  is 

task  type  dependent  is 

entry  e; 
end  dependent; 


type  dependent_ptr  is  access  dependent; 


task  body  dependent  is 
begin 
loop 
select 

accept  e  do 

end  e; 
or 

terminate; 
end  select; 
end  loop; 
end  dependent; 

tl  :  dependent;  --  direct  dependent  of  master 
begin  --  body  of  master. 

inner:  declare 

t2  :  dependent;  --  direct  dependent  of  inner. 
t3  :  dependent_ptr; 
begin 

~  direct  dependent  of  master. 
t3  :=  new  dependent; 

loop 
select 

accept  em  do 

end  em; 
or 

terminate;  --  Master. 
end  select; 
end  loop; 

end  inner; 
end  master; 

3.  A  Necessary  And  Sufficient  Condition  For  Termination 

We  introduce  the  notion  of  quiescence,  and  then  prove  a  condition  on  dependency 
trees  that  enables  a  distributed  implementation  of  termination. 

Definition:  If  r  is  a  task  with  no  dependents,  then  r  is  quiescent  if  t  is  complete  or  if 
t  is  waiting  at  a  terminate  alternative.  If  t  has  dependents,  then  t  is  quiescent  if  t  is  com- 
plete or  if  /  is  waiting  on  a  terminate  alternative,  and  all  dependents  of  t  are  quiescent.  A 
task  which  is  neither  quiescent  nor  terminated  is  said  to  be  active.  Finally,  if  t  is  a  mas- 
ter block  (i.e.  a  block  or  subprogram  with  dependent  tasks),  then  b  is  quiescent  if  b  is 
complete  and  all  dependents  of  b  are  quiescent. 


Note  that  if  a  task  t  is  quiescent  and  waiting  on  a  terminate  alternative,  it  is  not  valid 
to  terminate  it,  since,  for  example,  its  master  may  not  be  complete  and  may  still  call  t. 
We  can,  however,  make  the  following  claim. 

Proposition  1.  If  a  master  m  is  quiescent  and  complete  (i.e.  not  waiting  at  a  ter- 
minate alternative),  then  m  may  be  terminated. 

Proof.  Because  of  the  recursive  definition  of  quiescence,  any  task  that  depends  on  m 
is  either  complete  or  waiting  at  a  terminate  alternative.  Thus  m  may  be  terminated 
according  to  [LRM83J9.4.  D 

The  following  proposition  allows  us  to  implement  termination  in  a  simple  distri- 
buted fashion. 

Proposition  2.  While  a  master  m  is  quiescent,  no  dependents  of  m  can  be  called 

and  no  new  dependents  of  m  can  be  created. 

Therefore,  the  only  way  for  m  to  become  active  (assuming  m  is  waiting  on  a  ter- 
minate alternative)  is  for  another  task  to  invoke  m  directly.  The  implication  of  Proposi- 
tion 2  is  that  once  a  termination  wave  has  begun,  no  task  from  outside  the  wave  may  call 
a  task  being  terminated  in  the  wave.  Thus  it  is  not  necessary  for  the  tasking  system  to 
lock  out  other  tasks  during  a  termination  wave,  i.e.  for  a  termination  wave  to  be  imple- 
mented as  an  indivisible  operation.  Furthermore,  the  bookkeeping  needed  to  check  for 
and  initiate  termination  can  be  performed  in  a  relatively  local  manner,  as  described  in 
Section  4.  We  first  give  a  proof  of  Proposition  2. 

Proof.  We  will  prove  the  proposition  in  two  parts:  first  the  absence  of  callers  and 
then  the  absence  of  new  dependents. 

No  Dependents  Can  Be  Called.  Let  f  be  a  dependent  of  a  quiescent  master  m,  and 
suppose  there  is  another  task  t]  that  invokes  t.  Then  t]  cannot  be  a  dependent  of  m,  since 
all  dependents  of  m  are  quiescent.  First,  assume  that  t  is  a  task  object.  For  tl  to  call  r,  it 
must  be  the  case  that  m,  or  a  dependent  of  m  that  can  access  r,  passes  r  to  tl  through  a 
sequence  of  one  or  more  entry  calls  to  a  task  that  is  not  in  the  dependency  tree  rooted  at 
m.  (If  m  is  the  direct  master,  t  is  declared  in  m,  and  is  not  otherwise  accessible  by  name 
outside  the  dependency  tree  of  m.  If  m  is  not  the  direct  master,  /  is  declared  in  an  inner 
block  or  dependent  task  of  m.  All  dependents  of  the  direct  master  of  t  are  also  dependents 
of  m,  and  so  again  r  is  not  accessible  by  name  outside  the  dependency  tree  of  m.)  Let  o 
be  the  task  that  depends  on  m  and  passes  t  to  an  outer  dependency  tree.  Task  objects  are 
of  limited  type;  therefore,  r  may  not  be  passed  by  assignment  (e.g.  to  a  global  variable). 
For  tl  to  be  able  to  reference  t  (passed  as  an  in  parameter),  it  must  therefore  be  the  case 
that  o  is  currently  executing  an  entry  call,  and  so  cannot  be  quiescent  (See  Figure  5).  This 
is  a  contradiction. 


^A  select  statement  may  contain  a  terminate  aliemaiive  and  an  accept  alternative  for  an  interrupt  en- 
try. Such  an  accept  may  be  called  by  the  device  associated  with  the  enU7  while  the  master  of  the  task 
which  contains  the  select  is  quiescent.  However,  an  implementation  is  permitted  to  impose  further  requir- 
erments  for  the  selection  of  the  terminate  alternative  of  a  select  with  an  accept  alternative  for  an  interrupt 
entry.  We  can  either  consider  devices  associated  with  interrupt  entries  as  active  dependents  (requiring  that 
tasks  which  contain  interrupt  enunes  be  explicitly  aborted),  or  simply  ignore  interrupt  enuies  when  deter- 
mining if  the  terminate  alternative  should  be  selected. 


no_term(t)  =  if  r  is  complete  or  waiting  on  a  terminate 
alternative  then 

the  number  of  active  direct  and  inner  dependents  of  r 
else 

the  number  of  active  direct  and  inner  dependents  of  r  +  1. 

Thus  nojerm  for  a  task  includes  the  task  itself  in  the  count  of  active  dependents.  The 
extra  1  in  nojerm  when  a  master  is  not  complete  or  waiting  on  a  terminate  alternative 
allows  us  to  implement  the  test  for  quiescence  as  an  indivisible  operation.  Inner  depen- 
dents are  also  counted,  to  handle  the  case  illustrated  by  Example  4,  Section  2.  For  a  mas- 
ter block  nojerm  is  defined  as: 

nojerm(b)  =  if  ft  is  complete  then 

the  number  of  active  direct  dependents  of  b 
else 

the  number  of  active  direct  dependents  of  ft  +  1 . 

It  follows  from  Proposition  1,  for  a  master  m,  that  when  m  is  complete  and  nojerm(m)  = 
0,  m  may  be  terminated. 

b)  The  control  block  of  each  task  contains,  in  addition  to  a  nojerm  counter,  pointers  to 
its  master  task  and  master  block,  and  a  status  field.  The  status  of  a  task  is  set  only  by  the 
task  itself;  other  tasks  inspect  the  status  of  a  task  when  they  want  to  engage  in  an  activity 
with  that  task.  The  possible  values  of  status,  used  below,  are  self-explanatory.  Each 
block  or  subprogram  which  is  a  master  construct  also  contains  a  nojerm  counter. 

c)  An  additional  flag  first,  initially  0,  is  also  needed  in  a  master  task  control  block  to 
ensure  that  only  a  single  task  will  discover  that  a  master  task  can  terminate  (see  section 
4.3). 

The  next  subsections  present  the  detailed  implementation  of  termination.  The  only 
primitive  indivisible  instructions  used  below  are  simple  (but  unconventional)  increment 
and  decrement  instructions.  We  will  use  inc(v)  and  dec(v)  to  designate  them.  The  opera- 
tion inc(v)  (resp.  dev(v))  is  an  indivisible  instruction  which  simultaneously  increments 
(resp.  decrements)  v  by  1  and  returns  the  value  of  v  before  the  operation.  This  somewhat 
unusual  choice  of  primitives  is  influenced  by  the  existence  of  the  atomic  fetch-and-add 
operation  on  the  Ultracomputer  ([GLR83],  [Pfi85]),  but  can  be  implemented  on  other 
architectures  as  well.  The  advantage  of  the  fetch-and-add  operation  is  that  it  implements 
inc  and  dec  in  such  a  way  that  the  time  for  many  processors  to  simultaneously  perform  an 
inc(v)  or  dec(v)  operation  on  the  same  variable  v  is  the  same  as  for  a  single  processor. 
(Thus  serial  bonlenecks  that  might  otherwise  be  incurred  from  the  inc  and  dec  operations 
used  below  are  eliminated  on  an  Ultracomputer.) 

In  addition,  the  following  code  assumes  the  existence  of  a  (distributed)  scheduler 
that  manages  various  queues.  The  only  actions  involving  the  scheduler  below  are  calls  to 
block,  unblock  and  clear jinblock.  block,  unblock  and  clear jinblock  are  implemented 
with  a  counter,  numjevent.  When  a  task  self  executes  a  block(self),  self.numjvent  is 
decremented;  if  self. numevent  is  still  positive  then  ^e// continues  to  run.   When  a  task 
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Figure  5 


Next,  assume  t  was  created  by  the  evaluation  of  an  allocator.  Task  pointers  are  not 
limited  and  can  be  assigned.  Hence  a  task  t]  that  receives  a  pointer  to  r  as  an  in  parame- 
ter can  call  t  even  after  the  rendezvous  is  completed.  However,  we  claim  that  if  a  task  tl 
calls  t,  tl  must  be  a  dependent  of  its  direct  master.  This  follows  from  the  visibility  rules 
of  Ada.  By  definition,  if  m'  is  the  direct  master  of  t,  its  access  type  declaration  appears  in 
m  .  For  tl  to  call  t  it  must  be  able  to  see  the  access  type  declaration,  and  therefore  the 
type  of  tl  must  be  declared  within  the  scope  of  the  direct  master  of  t.  It  follows  that  tl  is 
a  dependent  of  m,  and  must  be  quiescent.  Therefore  no  dependents  of  m  can  be  called 
while  m  is  quiescent. 

No  Dependents  Can  Be  Created.  Suppose  that  a  task  tl  creates  a  task  t,  which 
becomes  a  dependent  of  a  quiescent  master  m.  As  above,  the  corresponding  access  type 
is  declared  in  the  direct  master  of  r,  and  the  access  type  definition  must  be  visible  to  tl . 
Thus,  tl  must  be  a  dependent  of  the  direct  master  of  r,  and  hence  be  a  dependent  of  m. 
By  hypothesis,  such  a  task  must  be  quiescent,  and  therefore  cannot  be  executing  an  allo- 
cator. Thus  while  m  is  quiescent,  no  new  dependent  can  be  created.  This  completes  the 
proof  of  Proposition  2.  D 

4.  Termination  On  An  MIMD  Architecture. 

4.1.  Data  Structures 

a)  To  implement  distributed  termination,  we  associate  a  counter  nojerm  with  each  mas- 
ter to  keep  track  of  the  number  of  its  direct  (and  inner)  dependents  that  are  not  quiescent. 
(Similar  counters  have  been  used  in  uniprocessor  implementations,  e.g.  [RB84].)  It  is 
clear  from  the  definition  of  quiescence,  that  if  all  direct  (and  inner)  tasks  are  quiescent, 
then  all  dependents  are  also  quiescent. 

For  a  master  task  no  term  is  defined  as: 


executes  an  unblock(t),  t.numevent  is  incremented;  if  t.numevent  was  -1,  the  task 
makes  t  ready  to  run.  When  a  task  5e// executes  a  clear _unblock(self),  self.num_event  is 
simply  decremented. 

4.2.  Termination  of  Blocks  and  Subprograms 

In  this  section  we  deal  only  with  the  termination  of  master  blocks  and  subprograms, 
that  is,  blocks  and  subprograms  that  have  dependent  tasks.  Because  we  are  only  dealing 
with  master  blocks,  the  description  of  some  of  the  actions  performed  when  tasks  are 
activated,  become  quiescent,  etc.,  is  omitted.  Termination  of  tasks  is  somewhat  more 
complicated,  and  is  described  in  more  detail  in  the  next  section.  The  nojerm  counter  in 
the  activation  record  of  the  master  block  indicates  when  the  block  can  be  exited:  a  brief 
summary  of  the  actions  performed  on  nojerm  is  first  specified,  and  code  to  implement 
this  specification  follows. 

When  a  block  b  is  entered  its  nojerm  is  set  to  1.  When  a  task  that  depends  directly 
on  b  is  activated,  it  increments  nojerm(b).  Similarly,  when  a  task  directly  dependent  on 
b  accepts  an  entry  call  while  waiting  on  a  terminate  alternative,  and  the  task  was  quies- 
cent before  the  call,  it  increments  nojerm(b). 

When  a  block  b  is  complete,  its  nojerm  is  decremented.  If  nojerm(b)  is  0,  then  b 
is  quiescent  and  can  be  terminated.  When  a  dependent  task  t  becomes  quiescent,  (either 
by  completing  or  wailing  on  a  terminate  alternative,  and  decrementing  its  own  counter  to 
0),  it  decrements  nojerm(b).  If  t  discovers  that  its  master  block  b  is  now  quiescent,  i.e. 
nojerm(b)  is  0,  it  notifies  its  master  task  (which  is  of  course  executing  b)  to  resume  and 
begin  a  termination  wave  for  b. 

Thus  a  master  block  b  can  terminate  when  b  is  complete  and  nojerm(b)  becomes  0. 
Before  terminating,  b  tells  all  its  direct  dependents  that  have  not  already  terminated  to 
terminate.  Each  direct  dependent  does  the  same  recursively. 

The  code  to  implement  the  termination  of  masters  that  are  blocks  or  subprograms  is 
given  below.  The  variable  5e// designates  the  task  executing  the  code. 

(1)  When  a  master  block  B  is  entered,  but  before  its  declarations  are  elaborated: 

—  no  active  dependents,  but  not  complete. 
b.NO.TERM  :=  1; 

(2)  When  a  master  block  B  completes: 

ifdec(B.NO_TERM)/=  Ithen 

—  Wait  for  dependents 

—  to  become  quiescent 

block(selO; 
end  if; 


—  Upon  awakening,  start  a  terminate  wave 
for  each  direct  dependent  T  of  B  loop 

if  T.STATUS  =  SELECT_TERM  then 

unblock(T); 
end  if; 
end  loop; 

—  Exit  from  block. 

(3)  When  a  task  becomes  quiescent: 

if  dec(self.MASTER_BLOCK.NO_TERM)  =  1  then 

—  The  master  is  quiescent.  Awaken  master 

—  task  to  start  termination  wave  of  the 
--  direct  dependents  of  the  block. 
unblock(self.MASTER_TASK); 

end  if; 

(4)  When  a  task  becomes  active: 

inc(self.MASTER_BLOCK.NO_TERM); 

4.3.  Termination  of  Master  Tasks 

The  operations  involved  in  task  termination  are  similar  to  those  for  block  and  sub- 
program termination.  There  are,  however,  more  complications.  Once  a  block  or  subpro- 
gram becomes  quiescent  it  cannot  become  active  again.  In  contrast,  a  quiescent  task  that 
is  waiting  on  a  terminate  alternative  may  become  active  by  being  called  by  a  master, 
sibling,  or  sibling  dependent.  Also,  unlike  a  block,  a  task  has  masters  that  must  be 
updated  when  the  status  of  a  task  changes.  The  nojerm  counter  of  a  task  /  is  incre- 
mented as  follows: 

(i)     When  t  is  activated,  it  sets  its  nojerm  to  1. 

(ii)    If  t  is  waiting  on  a  terminate  alternative  and  enters  a  rendezvous,  it  increments 
no_term. 

(iii)  Finally,  no_term(t)  is  incremented  by  a  direct  or  inner  dependent  when  the  depen- 
dent is  activated  or  when  the  dependent  changes  from  quiescent  to  active. 

A  significant  observation  is  that  when  a  quiescent  task  becomes  active  (or  a  task  is 
created),  it  is  only  necessary  to  propagate  the  information  one  level  up  a  def>endency  tree 
-  i.e  to  its  master  task  (and/or  master  block).  This  follows  from  Proposition  2:  if  a  task  t 
is  invoked,  it  must  be  the  case  that  its  master  task  is  not  quiescent,  and  hence  no  change 
of  state  (from  quiescent  to  active)  can  occur  in  the  master  task.  Consequently,  no  further 
updating  nor  checking  need  be  performed.  A  similar  remark  applies  when  a  task  is 
dynamically  created. 

Counters  are  decremented  as  follows.  When  t  completes  or  waits  on  a  terminate 
alternative,  it  decrements  its  nojerm.  If  in  so  doing  it  becomes  quiescent,  it  also  decre- 
ments nojerm  of  its  master  task.  When  a  task  t  discovers  that  its  master  task  m  is  now 
quiescent  and  m  is  complete,  then  t  notifies  m  to  begin  a  termination  wave;  \i  m  is  waiting 
on  a  terminate  alternative,  then  recursively  t  decrements  masters  of  its  master  task  until 
either  of  the  following  occurs: 


(1)  t  finds  a  master  task  that  is  not  quiescent  (in  which  case  t  either  blocks  if  it  is  not 
complete  or  terminates  its  own  dependents  if  it  is  complete), 

(2)  r  finds  a  master  task  that  is  quiescent  and  complete,  and  can  therefore  start  a 
downward  termination  wave. 

It  is  possible  that  more  than  one  task  finds  that  a  master  task  is  quiescent  and  com- 
plete.^ To  ensure  that  the  termination  wave  is  initiated  only  once  the^r^r  flag  of  the  mas- 
ter task  is  used.  After  a  task  discovers  that  a  master  task  is  quiescent  and  complete  it 
increments  the  first  flag  of  the  master  task.  If  first  was  0,  then  the  task  initiates  the  termi- 
nation wave. 

The  recursive  decrementing  of  noterm  and  checking  for  quiescence  is  the  only 
time  that  information  must  be  propagated  for  more  than  one  level  in  a  dependency  tree. 
It  need  not,  however,  be  an  indivisible  operation,  nor  be  performed  in  a  critical  section. 
Furthermore,  propagation  and  checking  need  only  be  performed  up  a  single  branch  in  the 
dependency  tree.  No  subsequent  rechecking  down  the  tree  is  required.  These  observa- 
tions follow  from  Proposition  2:  while  a  task  t  is  updating  an  ancestor  m  and  m  is  quies- 
cent, the  status  of  all  the  dependents  of  m  cannot  change. 

However,  a  termination  wave  is  a  complex  affair;  the  task  that  starts  the  wave  (the 
master  task  awakened  by  step  2  above)  must  both  recursively  update  its  masters' 
nojerms  since  it  has  become  quiescent,  and  notify  its  dependents  to  terminate. 

Implementation  details  follow.  The  code  presented  is  for  tasks  that  have  both  mas- 
ter blocks  and  master  tasks.  If  a  task  has  no  master  block,  the  code  is  simpler. 

(1)  When  a  task  T  is  activated,  prior  to  activating  any  tasks  declared  in  T: 

self.NO_TERM  :=  1; 
inc(self.MASTER_TASK.NO_TERM); 
inc(self.MASTER_BLOCK.NO_TERM); 
self. STATUS  :=  ACTIVE; 

(2)  Upon  entering  a  rendezvous  in  a  selective  wait  with  a  terminate  alternative: 


^o  see  how  this  can  occur,  consider  Example  3  in  Section  2.  Suppose  task  (2  completes,  finds  iis 
master  dependent  waiting  to  terminate,  and  then  decrements  the  counter  of  master.  Suppose  master  is 
waiting  on  a  terminate  alternative,  and  that  its  counter  becomes  0.  Then,  another  task  awakens  master, 
which  then  calls  entry  e  of  dependent,  and  dependent  creates  a  new  task  t3.  Next  master  completes,  t3 
completes,  finds  dependent  quiescent,  and  updates  the  counter  of  master.  If  t2  has  done  nothing  in  the 
meantime,  both  i2  and  t3  will  find  master  complete  and  quiescent,  and  hence,  signal  master  to  start  a  termi- 
nation wave.  In  a  similar  scenario,  it  is  also  possible  that  both  a  master  task  and  a  dependent  find  that  the 
master  is  quiescent  and  complete. 


if  inc(self.NO_TERM)  =  0  then 

—  Was  quiescent  before  rendezvous. 

—  Restore  masters'  NO_TERM  counts. 
inc(self.MASTER_TASK.NO_TERM); 
inc(self.MASTER_BLOCK.NO_TERM); 

end  if; 

self.STATUS  :=  ACTIVE; 

(3)  Upon  reaching  a  selective  wait  with  a  terminate  alternative  and  no  acceptable 
calls: 

self.STATUS  :=  SELECT_TERM; 
if  dec(self.NO_TERM)  =  1  then 

—  Am  quiescent  now. 

—  Propagate  information  up  tree. 
CHECK_TERMlNATION(self.MASTER_TASK, 

self.MASTER_BLOCK); 
end  if; 

—  Will  be  awakened  either  by  a  rendezvous  or  by  a 
--  termination  wave  from  above. 

block(selO; 

(4)  When  a  task  completes: 

self.STATUS  :=  COMPLETE; 
ifdec(self.NO_TERM)/=  1  then 

--  Wait  for  dependents  to  be  quiescent. 

block(selO; 
else 

if  inc(self.nRST)  >  0  then 

--  A  dependent  also  has  discovered  that  self  is 

—  quiescent  and  complete,  so  clear  pending  unblock. 
clear_unblock(self); 

end  if; 
end  if; 

—  Am  quiescent  now. 

—  Update  and  check  master(s). 
CHECK_TERMINATION(self.MASTER_TASK, 

self.MASTER_BLOCK); 
--  Notify  dependents  to  terminate. 
for  each  direct  and  inner  dependent  T  of  self  loop 
if  T.STATUS  =  SELECT_TERM  then 

unblock(T); 
end  if; 
end  loop; 

—  Terminate  self. 


(5)       Recursive  subprogram  to  update  and  check  master  tasks  and  block. 

procedure  CHECK_TERMINATION(M,  B)  is 

—  called  by  a  quiescent  dependent  of  M  and  B. 
begin 

if  M  /=  null  and  then  dec(M.NO_TERM)  =  1  then 

—  Master  task  is  quiescent. 

if  M.STATUS  =  COMPLETE  then 
--  Must  check  quiescence  again  to 

—  avoid  potential  race  condition.'* 

ifM.NO_TERM  =0  and  then 

—  M  is  quiescent  and  complete. 

—  Check  for  first  caller. 
inc(M.nRST)  =  0  then 

—  Start  termination  wave. 
unblock(M); 

end  if; 
else 

--  M  is  waiting  on  terminate  alternative. 

—  Check  M's  master. 
CHECK_TERMINATION(M.MASTER_TASK, 

M.MASTER_BLOCK); 
end  if; 
end  if; 

—  If  the  direct  master  of  self  is  a  block  B, 

—  update  and  check  B. 

if  B  /=  null  and  then  dec(B.NO_TERM)  =  1  then 

—  B  is  quiescent,  (see  Section  4.1) 

—  M  is  executing  B,  and  thus  not  quiescent 
unblock(M); 

end  if; 
end  CHECK_TERMINATION; 

4.4.  Absence  of  Race  Conditions 

We  must  show  that  there  are  no  race  conditions  which  might  lead  to  premature  ter- 
mination, duplicate  detection  of  termination  or  failure  to  detect  when  the  conditions  for 
termination  are  met.    The  potential  for  race  conditions  exist  because  more  than  one 


"a  race  condition  is  possible  here  because  the  check  for  quiescence  and  completion  is  not  an  indivisi- 
ble operation.  After  the  first  lesl  for  quiescence  but  before  M.STATUS  is  checked,  M  may  be  invoked  and 
a  dependent  Tl  of  M  activated.  If  M  subsequently  completes,  the  second  test  (for  completeness)  will 
succeed,  even  though  M  is  no  longer  quiescent.  Therefore,  an  additional  test  for  quiescence  is  necessary. 
See  Section  4.3. 


dependent  task  may  access  a  task  control  block  concurrendy,  and  some  of  the  actions 
involved  in  bookkeeping  are  not  indivisible.  We  will  discuss  each  of  the  cases  separately 
below. 

4.4.1.  Premature  Termination  Waves 

A  potential  race  condition  involves  the  checks  that  the  actual  termination  conditions 
for  initiating  a  termination  wave,  completeness  and  quiescence,  are  met.  These  are  two 
separate  checks,  and  hence  not  an  indivisible  operation.  These  checks  are  performed 
when  a  dependent  decrements  the  counter  of  a  master  task  (in  procedure 
CHECK_TERMINATION),  and  when  a  master  task  completes  and  checks  its  own 
counter.  In  the  first  case,  the  dependent  checks  for  nojerm  =  0  twice.  Therefore,  if  the 
state  changes  between  the  check  of  noterm  and  the  check  for  completeness,  the  depen- 
dent will  not  initiate  a  termination  wave  prematurely.  (Note  that  once  a  task  is  complete 
its  status  cannot  change.)  In  the  latter  case,  the  master  task  sets  its  status  to  complete 
before  decrementing  its  nojerm  counter.  Thus,  either  the  master  or  the  dependent  will 
discover  correctly  the  case  in  which  a  termination  wave  can  begin. 

Given  that  either  a  master  task  m  or  one  of  its  dependents  correctly  discover  when  m 
is  complete  and  no_term(m)  is  0,  we  must  prove  that  when  these  conditions  are  met  a  ter- 
mination wave  of  the  dependents  of  m  can  be  initiated.  The  following  proposition  is  use- 
ful in  showing  that  a  termination  wave  will  not  be  started  prematurely. 

Proposition  3.  While  a  task  t  is  executing  code  fragments  (1)  or  (2),  nojerm  of  its 
master  cannot  become  0. 

Proof.  This  follows  direcdy  from  the  contraposirive  of  Proposition  2:  While  depen- 
dents of  a  master  m  are  being  called  or  new  dependents  of  m  are  being  created,  m  cannot 
be  quiescent.  D 

A  termination  wave  of  the  dependents  of  m  will  be  initiated  after  0  or  more  calls  to 
CHECK_TERMINATE  are  made  by  the  task  that  initiates  the  termination  wave.  We 
will  prove  that  no  termination  waves  are  started  prematurely  by  induction  on  the  number 
of  tiiese  calls  to  CHECK_TERMINATE. 

Basis.  When  /  is  0,  then  it  is  m  that  discovers  that  a  termination  wave  should  be  ini- 
tiated. From  Proposition  1  we  can  deduce  that  a  termination  wave  can  be  initiated. 

Induction  Hypothesis.  When  it  is  discovered  that  a  master  task  m  is  quiescent,  i.e. 
m.status  is  complete  and  nojerm(m)  =  0,  via  /  calls  to  CHECK_TERMINATE  then  a 
termination  wave  of  the  tasks  in  the  dependency  tree  rooted  at  m  can  be  initiated. 

Step.  Suppose  the  termination  wave  is  discovered  by  a  task  t  via  /-i-7  recursive  calls 
of  CHECK_TERMINATE.  While  /  is  active  we  can  deduce  from  Proposition  3  that 
nojerm(m)  is  not  0,  and  hence,  no  termination  wave  will  be  initiated  before  t  calls 
CHECK_TERMINATE.  The  first  call  made  by  t  to  CHECK_TERMINATE  must  find 
Lmasterjask  quiescent,  since  we  know  i  eventually  initiates  a  termination  wave.  From 
Proposition  2  we  can  deduce  that  while  t.master  is  quiescent  no  dependents  of  t.master 
will  be  called  or  created,  and  hence,  disturb  the  conditions  for  initiating  a  termination 
wave.  We  complete  the  proof  by  appealing  to  the  Induction  Hypothesis  for  next  /  calls. 
D 


4.4.2.  Multiple  Detections  of  Termination 

A  termination  wave  is  discovered  by  either  a  master  task  m  or  one  of  its  dependents. 
At  most  one  dependent  will  succeed  in  discovering  that  the  conditions  for  initiating  a  ter- 
mination wave  and  incrementing  m.first  to  1.  When  m  completes  and  finds  nojerm(m)  is 
0,  it  also  increments  m.first;  if  m  first  was  already  incremented  then  a  dependent  has  also 
discovered  that  m  is  quiescent,  so  m  clears  the  pending  unblock,  otherwise  m  has  suc- 
ceeded in  inhibiting  a  dependent  from  executing  the  unblock(m).  Therefore,  exactly  one 
dependent  or  master  task  m  will  initiate  a  termination  wave  of  the  tasks  in  the  depen- 
dency tree  rooted  at  m. 

4.4.3.  Failure  to  Detect  Termination 

Every  time  noterm  of  a  task  is  decremented  it  is  tested  for  being  0.  When  a  depen- 
dent task  that  is  about  to  wait  on  a  terminate  alternative  finds  its  noterm  =  0,  it  calls 
CHECK_TERMINATE  to  test  its  masters.  In  CHECK_TERMINATE  a  dependent  will 
check  twice  if  the  noterm  of  a  master  task  is  0;  before  and  after  it  checks  if  the  status  of 
the  master  task  is  complete.  A  master  alway  sets  its  status  prior  to  decrementing  or 
incrementing  its  noterm  count.  In  panicular  a  master  task  sets  its  status  to  complete 
before  decrementing  its  nojerm  counter.  Thus  either  the  dependent  or  the  master  task 
will  detect  when  a  termination  wave  can  be  initiated. 

5.  Conclusion 

A  simple  implementation  of  task  termination  in  an  environment  in  which  tasks  can 
be  nested  and  created  dynamically  uses  counters:  the  counter  maintains  the  number  of 
subtasks  of  a  given  task,  and  the  last  subtask  to  decrement  the  counter  initiates  termina- 
tion. We  have  shown  that  this  implementation  strategy  can  be  used  for  Ada  tasks  in 
multiprocessor  environment  with  shared  memory,  in  spite  of  the  complexities  of  the  Ada 
tasking  model.  This  scheme  has  been  incorporated  into  a  full  design  of  the  ada  tasking 
model,  which  is  currently  being  implemented. 

Furthermore,  our  distributed  implementation  may  be  adapted  for  an  architecture 
without  shared  memor>',  in  which  message  passing  must  be  used.  In  such  an  environ- 
ment, when  a  dependent  task  becomes  quiescent  or  active,  it  must  send  a  message  to  its 
master  to  update  its  nojerm  counter.  With  our  implementation,  fewer  messages  need  by 
exchanged  than  in  other  proposed  implementations,  e.g.  [Kaf|. 
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APPENDIX  -  LIBRARY  PACKAGES 

A  task  t  can  gain  visibility  to  a  task  pointer  definition  if  the  task  pointer  type  is 
declared  in  a  library  package  and  t  names  the  library  package  in  a  with  clause.  The  rules 
of  Ada  specify  that  any  library  units  named  within  a  main  program  or  its  subunits  are  ela- 
borated before  the  execution  of  the  main  program.  The  rules  for  task  dependencies  state 
that  a  task  created  by  the  evaluation  of  an  allocator  directly  depends  on  the  master  that 
elaborated  the  corresponding  access  type  definition.  Thus,  any  tasks  created  by  the 
evaluation  of  an  allocator  whose  type  definition  appears  in  a  library  package  directly 
depend  on  the  library  package.  As  a  consequence,  such  tasks,  as  direct  dependents  of 
library  packages,  can  be  passed  to  the  direct  dependents  of  the  main  program.  Further- 
more, new  direct  dependents  of  the  library  package  can  be  created  by  the  direct  depen- 
dents of  the  main  program.  It  would  seem  that  Proposition  2  is  not  true  for  tasks  that  are 
dependent  on  library  packages. 

However,  the  reference  manual  states  in  9.4(13),  termination  of  the  main  program 
awaits  termination  of  any  dependent  task  even  if  the  corresponding  task  type  is  declared 
in  a  library  package.  On  the  other  hand,  termination  of  the  main  program  does  not  await 
termination  of  tasks  that  depend  on  library  packages;  the  language  does  not  define 
whether  such  tasks  are  required  to  terminate.  So  the  main  program  need  no  be  con- 
cerned with  the  termination  of  library  packages  and  their  dependents. 

In  order  to  handle  termination  in  a  uniform  manner,  an  implementation  could  have 
an  outermost  task  on  which  the  main  program  and  library  packages  are  dependent.  The 
task  could  either  wait  for  the  tasks  that  are  dependent  on  the  library  packages  to  ter- 
minate or  cease  to  exist  when  the  main  program  finished  execution.  Either  solution 
adheres  to  the  reference  manual,  which  does  not  require  that  tasks  that  depend  on  library 
packages  terminate,  and  the  termination  of  the  main  program  is  in  no  way  affected. 
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